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1.0  INTRODUCTION 

The  development  of  a  computer  based  model  for  the  evaluation 
of  deck  plan  layouts  for  the  Navy  has  resulted  in  the  LAYEVAL  evaluation 
system.  It  represents  a  procedure  for  assessing  selected  importance 
weights  assigned  by  decision  makers  in  the  layout  evaluation  process. 

This  model  is  described  in  detail  in  [1]  and  a  specification  for  the 
computer  system  appears  in  [3].  Figure  1  displays  the  conceptual  system 
and  its  components.  The  outlined  section  contains  the  essential 
elements  that  comprise  the  LAYEVAL  system. 

The  LAYEVAL  system  has  yet  to  be  thoroughly  tested,  although 
preliminary  validation  efforts  have  been  undertaken  and  these  results 
are  described  in  this  report.  It  is  anticipated  that  a  more  complete 
validation  will  be  conducted  in  a  later  effort. 

Several  experimental  directions  were  explored  during  the  testing 
of  LAYEVAL.  These  directions  were  not  visible  to  the  surrogate  decision¬ 
maker  (DM)  who  is  shown  as  the  "Designer"  in  Figure  1.  The  DM  was  asked 
to  supply  specific  information  and,  later,  was  shown  the  weights  that 
resulted  from  his  data.  The  various  data  translations  and  optimization 
efforts  were  not  displayed  to  the  DM,  but  these  did  constitute  the 
areas  of  exploration  for  research  purposes.  ^_The  DM  interacted  with  a 
hard  copy  questionnaire  documented  in  [2].  The  results  of  this 
interaction  and  the  experiments  on  translation  and  optimization  are 
reported  in  the  later  sections  of  this  document.  Since  the  experiments 
dealt  with  possible  alternative  formulations  of  the  mathematical  model, 
it  Is  appropriate  to  review  the  model  development  prior  to  presenting 
the  results.  This  next  section  presents  the  basic  mathematical  model 
with  additional  details  available  in  [1], 
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1.1  REVIEW  OF  MATHEMATICAL  MODEL 

LAYEVAL  is  based  upon  but  significantly  different  from,  a 
methodology  for  developing  value  functions  known  as  the  composite 
criterion  model  (CCM)  formulated  by  Srinivasan  and  Shocker  [4,5,6]. 

The  CCM  is  a  technique  for  analyzing  a  DM's  preferences  when  there 
are  multiple  conflicting  criteria.  In  the  context  of  layout  evaluation, 
for  the  1—  layout  alternative,  there  is  a  vector  [a  ^3  of  attribute 
values  for  UP,  the  set  of  attributes.  The  score  for  layout  i  is: 

s,  .-0  (!>,„])  •  J  Vlk  (1) 

In  the  CCM  methodology  a  limited  number  of  explicit  pairwise  comparisons 
is  used  to  reveal  the  proper  values  of  the  attribute  weights,  w^,  for 

a  given  decision  maker.  Note  that  the  value  function,  "J  (  ),  is 

\ 

assumed  to  be  linear, 

Given  two  layout  alternatives,  1  and  j,  if  the  DM  prefers  i  over  j 
then  Sj  i  3y  1,e, 

l  Vlk  *  l  Vjk  i  0 


°r  (2) 

l  (aik  "  ajk)wk  ^  0 

Thus,  given  a  set,  (2,  of  pairwise  layout  comparisons,  there  Is  a  corresponding 
set  of  constraints,  (2),  which  must  be  satisfied  by  the  weights,  w^. 
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Since  It  Is  possible  that  the  DM  may  err  In  the  preference  responses 
(or  that  a  linear  value  function  assumption  Is  not  completely  satisfied), 
the  feasible  set  defined  by  (2)  for  (1,  j)efl  may  be  empty.  In  that  case, 
it  Is  desired  to  determine  a  "best"  set  of  weights,  i.e.  one  that  comes 
closest  to  rationalizing  the  OM's  preferences,  where  closeness  is  rigorously 
defined. 

The  value  measurement  problem  considering  only  attribute  importance 
values  Is  to  determine  the  weights,  w^.  The  CCM  provides  a  means 
for  determining  the  weights,  specifically,  (minimizing  the  sum  of  the 
infeasibilities  in  constraints  (2),  by  solving  the  following  linear 

x 

programming  problem: 


Minimize  Z  c..Z., 

(1.  J)  1j 

(3) 

s.t.  Z  +  Zjj  >  0  for  (1#  j )efl 

keP 

(4) 

z  z  c  a  w.  -  i 

keP  (1,  j)efl  k 

(5) 

hi  to 

(1,  j)eO 

(6) 

wk  >  0 

kl’P1 

(7) 

wk  <  0 

kifP2 

(8) 

wk  urs 

kfPj 

(9) 

where: 


cf  j  *  confidence  coefficient  (c^  2  0)  associated  by  the  DM 

with  the  paired  comparison  judgement  (i,  j). 

P  ■  Set  of  attributes  *  P  »  P P2 ^  P3 *  ^i^Pj  *  0  for  i  ft  j 

aijk  *  aik  "  ajk 

-  r 

The  notion  of  weights  in  this  model  includes  both  the  units  in 
which  each  attribute  is  scaled  and  the  relative  importance  (or  salience) 
of  each  attribute  to  the  individual  in  the  context  of  his  multi -attribute 
decision-making.  The  combining  of  both  scale  factors  and  individual 
measures  of  salience  into  weights  has  been  used  by  other  researchers  [2], 

One  difficulty  in  using  the  CCM  directly  for  layout  evaluation 
is  the  potential  for  an  excessive  burden  upon  the  DM  concerning 
preferences  about  entire  layouts  and  the  need  to  express  such 
preferences  among  many  entire  layouts.  Another  problem  is  that  of 
optimization  complexity,  considering  the  larger  ship  designs  developed 
by  the  Navy,  since  direct  application  of  the  CCM  results  in  non¬ 
linear  modelsand  these  are  difficult  to  solve  in  practice.  Therefore, 
LAYEVAL  was  developed  to  overcome  these  difficulties  while  retaining 
the  essence  of  the  CCM  approach. 

An  Important  element  of  LAYEVAL  is  the  decomposition  of  the 
layout  Into  smaller  elements  where  activities  or  compartments  become 
the  focus  of  the  DM's  attention.  With  this  perspective,  two  types 
of  attributes  are  considered.  The  first  category,  the  intrinsic 
attributes,  attributes  that  describe  the  operating  conditions  of  the 
space  and  its  assigned  activity.  Temperature,  noise  and  humidity 


levels  are  examples  of  Intrinsic  attributes.  It  Is  further  assumed 

that  the  operating  conditions  of  a  space  are  primarily  affected  by  operating 

conditions  of  spaces  near  or  immediately  adjacent  to  it. 

The  second  category  Is  that  of  flow  attributes.  These  attributes 
are  distinguished  by  Interactions  between  specific  activities,  and  the 
distance  between  these  Interacting  activities  Is  an  important  considera¬ 
tion,  Material  flow,  personnel  Interaction  and  egress  are  examples  of 
flow  attributes. 

Although  Intrinsic  attributes  or  flow  attributes  may  have 
continuous  measurement  scales,  It  Is  further  conjectured  that  the  DM 
considers  only  ranges  of  the  attribute  value  rather  than  the  exact 
values.  Specific  design  requirements  in  some  cases  may  be  stated  in 
terms  of  ranges  of  these  attribute  values,  but  past  practice  or  con¬ 
vention  and  simply  ease  of  Interpretation  on  the  part  of  the  OM  may 
also  support  this  assumption. 

Thus  the  measurable  attributes  to  be  considered  In  the  layout 
evaluation  process  are  assumed  to  be  either  Intrinsic  or  flow  attributes. 
Further,  It  is  assumed  that  the  DM  Is  primarily  concerned  with  a  dis¬ 
crete  number  of  Intervals  of  attribute  values  during  evaluation.  The 
following  sections  demonstrate  that  layout  attributes  having  these 
characteristics  can  be  accomodated  In  the  CCM  methodology  in  a 
manner  that  significantly  eases  the  computational  burden. 
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1.1.1  A  VALUE  FUNCTION  FOR  INTRINSIC  ATTRIBUTES 

Consider  an  intrinsic  attribute  such  as  ambient  noise  level.  For 
each  activity,  a,  the  DM  can  specify  a  desired  or  acceptable  attribute 
level,  e.g.,  very  quiet,  quiet,  moderately  noisy,  or  very  noisy. 

Suppose  that  for  attribute  k,  there  are  L,,  such  intervals  or  attribute 
levels,  specified  by  the  DM,  and  that  1^  Is  the  desired  or  acceptable 
attribute  level  for  activity  a  on  attribute  k. 

Now  consider  actvltles  a  and  b,  adjacent  in  some  layout.  If  lk 
for  activity  a  does  not  equal  lk  for  activity  b,  then  one  of  the  activities 
may  not  attain  its  desired  or  acceptable  attribute  level.  For  example, 
placing  a  freight  elevator  next  to  a  conference  room  doesn't  affect 
the  acceptability  of  the  noise  level  in  the  freight  elevator,  but  it 
might  lead  to  an  unacceptable  or  undesirable  noise  level  in  the 
conference  room.  Thus,  for  intrinsic  attributes,  it  seems  reasonable 
to  focus  concern  on  the  differences  in  target  attribute  levels  between 
adjacent  activities. 

From  this  example.  It  is  clear  that  attribute  differences  between 
adjacent  activities  are  not  reflexive,  i.e.,  the  difference  doesn't 
matter  from  the  consideration  of  the  freight  elevator,  while  it  certainly 
does  matter  from  the  consideration  of  the  conference  room.  Therefore, 

If  there  are  attribute  levels,  then  there  are  lk  (L^  -  1)  +  1  distinct 
attribute  level  difference  categories. 

Given  a  specific  layout,  let  Xa(an  be  the  number  of  activities 
adjacent  to  activity  a  whose  specified  level  of  attribute  k  Induces 
an  attribute  difference  (relative  to  a's  specified  level,  lak)  In 
difference  category  m,  1  <  m  <,  L^  (L^  -  1)  +  1.  Then  the  attribute 
value  function  Is  given  by: 

sak  "  *  ^km  Xakm 

m 

-■ _  . 


(10) 
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This  value  function  permits  the  comparison  of  different  layouts 
considering  only  activity  a.  The  weights  y.^  must  be  determined. 

The  Sak  values  must  be  aggreg?  ted  in  some  way  to  develop  an 
overall  score,  Sk,  for  a  layout.  In  aggregating  the  Sak*  not  all 
activities  and  not  all  attributes  are  equally  important.  For  example, 
the  DM  may  place  greater  importance  on  achieving  the  target  noise 
level  in  the  conference  room  than  he  places  on  achieving  the  target 
in  the  freight  elevator.  Therefore,  suppose  the  DM  can  distinguish 
Ik  importance  levels  for  attribute  k  (e.g.,  unimportant,  important, 
critical).  Also,  let  1^  be  the  importance  attached  to  the  attribute 
k  level  for  activity  a,  specified  by  the  DM  (1  <  ik  <_  Ik).  The 
definition  of  Ik  and  the  specification  of  i^  are  for  attribute  k 
where  k  can  be  either  an  intrinsic  or  a  flow  attribute. 

For  every  activity,  a,  the  DM  has  specified  both  1^,  the 
target  attribute  level  for  an  intrinsic  attribute,  and  i^  ,  the 
Importance  level  associated  with  achieving  that  target.  The  aggregation 
of  the  Sak  values  should  reflect  these  attribute  level -importance 
level  pairs.  Therefore,  define  a  weight,  y^k>  that  applies  to  all 
activities  for  which  the  DM  specified  attribute  level  7  and  importance 
level  1.  These  weights  give  numerical  values  to  the  DM's  interval  or 
level  specifications.  Then  the  score  for  activity  a.,  in  a  specified 
layout  with  only  intrinsic  attributes,  .is  given  by  the  value  function 

sa  "  J  y11k  sak 
where  1  ■  1  k 

ak 


and  1*1 


This  value  function  permits  the  comparison  of  different  layouts 
considering  only  activity  a.  The  weights  y^  must  be  determined. 

1.1.2  A  VALUE  FUNCTION  FOR  FLOW  ATTIBUTES 

While  flow  and  Intrinsic  attributes  are  quite  different  in  nature, 
similar  methods  can  be  used  for  developing  their  respective  attribute 
value  functions.  For  each  type  of  flow,  f,  suppose  the  DM  can  dis¬ 
tinguish  Lf  distinct  levels  of  Interaction  and  can  assign  (nonreflexive) 
interaction  levels  lafa^  and  lbaf  for  the  flows  between  activities  a 
and  b. 

If,  in  a  particular  layout,  D(a,b)  is  the  distance,  in  any 
appropriate  metric,  between  activities  a  and  b,  then 

0U.b)  i„af 

Is  their  weighted  Interaction  for  flow  type  f  from  activity  b  to 
activity  a.  Even  within  flow  type  f,  all  Interactions  between  pairs 
of  activities  may  not  be  equally  Important,  As  before,  suppose  the 
DM  can  distinguish  1^  importance  levels  for  flow  type  f  and  assigns  the 
flow  Interaction  lba^  to  an  importance  category  i^.  The  specific  weight 
associated  with  this  Importance  category  is  u^  where  the  values  of 
u-jf  are  to  be  determined  by  the  model.  Thus,  the  weighted  Interaction 
for  flow  type  f  from  activity  b  to  activity  a  becomes, 

ulf  D(a,b) 

and  the  value  function  for  a  particular  activity  a  and  flow  attribute  f, 
considering  all'' other  activities,  Is  given  by: 

saf  "  2  ulf  °(a’b)  (12) 

b?*a 


where 
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The  CCM  methodology  can  be  used  to  determine  the  weights. 

Also,  once  the  weights,  u^,  are  known,  the  score  for  activity  a, 
in  a  specified  layout  considering  only  flow  attributes,  can  be 
obtained: 

sa  '  **11f  saf  <13> 

The  y.^  weight,  defined  previously,  relates  to  the  flows  themselves 
in  equation  (13),  while  In  equation  (11)  the  target  differences  were 
important. 


1.1.3  A  VALUE  FUNCTION  FOR  THE  LAYOUT 

The  weight,  y^,  that  applies  to  all  activities  for  which  the 
DM  specified  or  measured  attribute  level  1  and  importance  level  i 
necessarily  involves  consideration  of  each  activity  and  each 
attribute  In  its  specification.  Hence,  the  relative  Importance  of 
activities  and  their  attribute  levels  are  normalized  through  the  y^^ 
variable.  Thus,  In  a  manner  consistent  with  the  assumption  of  an 
additive  model,  the  score  for  the  entire  layout  can  be  otained  from 
the  following  value  function: 

5  ‘  ?  f.k  (y,lk  s*k  * y,lf  hf)  (u) 


•  #' 

I 


n 


2.0  BACKGROUND 

The  model  described  in  the  previous  section  was  tested  in  the 
context  of  ship  layouts.  The  DM  for  this  case  was  a  volunteer 
graduate  student  studying  ship  arrangement  design  who  had  previously 
served  in  the  Navy.  Initially,  the  questionnaire  was  administered 
to  three  volunteer  graduate  students.  Only  two  of  the  questionnaires 
were  returned.  Of  the  two,  one  seemed  to  have  responses  that  failed 
to  follow  directions  and  definitions.  Hence^'t  was  not  used  to  test 
the  model . 

The  limited  amount  of  the  volunteer  DM's  available  time 
necessitated  the  use  of  a  small  hypothetical  ship  consisting  of  26 
activities  and  6  decks  for  this  study.  Since  multiple  decks  were 
considered,  the  problem  was  three  dimensional. 
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Discussion  with  personnel  at  the  Naval  Sea  Systems  Command  helped 
to  decide  on  the  activities  to  be  included  in  the  layout  design,  as 
well  as  a  potential  set  of  attributes  to  be  considered  in  evaluating 
the  alternate  designs.  A  set  of  requirements  to  be  satisfied  was 
also  provided  by  the  Navy  personnel  and  this  enabled  the  generation  of 
four  layouts  for  comparison.  Each  layout  should  have  been  acceptable 
to  the  DM,  thus  permitting  evaluation  based  on  the  attributes  considered. 
As  indicated  earlier,  the  details  of  the  layouts  and  the  questionnaire 
appear  in  Report  No.  5  [  1]. 

Two  interaction  sessions  were  held  withtthe  DM,  the  first  of 
about  six  working  hours  was  used  to  explain  the  model  to  the  DM,  to 
discuss  the  attributes  used  for  evaluation,  and  to  obtain  the  responses 
necessary  for  determining  the  unknown  weights  (variables)  of  the  model. 


Bl>u. 


Because  of  the  limited  interaction  time  available,  only  three 
intrinsic  attributes  (noise,  temperature,  and  humidity),  and  two 
flow  attributes  (nearness  to  desired  decks  and  functional  performance) 
were  selected. 

The  second  session  of  about  two  hours  was  used  to  discuss  the 
results  obtained  from  the  first  session  responses  and  to  uncover  the 
causes  for  some  unexpected  results  displayed  by  the  model  solutions. 

In  the  first  session,  the  DM  was  directed  to  the  appropriate  sections 
of  the  questionnaire  and  the  questions  were  fully  explained.  Members 
of  the  research  team  were  always  available  during  the  session  for  any 
additional  clarification. 

The  Multiple  Purpose  Optimization 'System  (MPOS)  package  from 
Northwestern  University  was  used  to  obtain  the  solution  to  the 
optimization  problems  (necessary  in  the  stages  of  LAYEVAL),  on  the  CDC 
Cyber  70  series  computer  of  the  University  of  Massachusetts.  Solution 
times  were  of  the  order  of  5-15  seconds,  including  compilation  and 
execution  times,  and  thus  the  model  appears  to  be  Quite  efficient  in 
the  use  of  computing  resources. 

Table  I  displays  the  expressed  preferences  by  the  DM  for  the  levels 
of  intrinsic  attributes  and  the  layout  activities.  The  individual  entries 
in  the  Table  reflect  both  the  specification  of  preference  and  the 
confidence  with  which  the  DM  held  the  preference.  Table  II  displays 
similar  data  for  the  preferences  for  layouts  expressed  by  the  DM  when 
considering  all  attributes  combined.  The  indication  of  Stage  1  and 
Stage  2  refers  to  the  phases  of  the  LAYEVAL  solution  procedure  in 
which  these  data  utilized.  The  flow  attribute  data  was  not  used  for 
reasons  to  be  developed  later. 
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Note:  Entry  1,2  indicates  that  the  first  layout  of  the  pair  (hence  1)  is 
preferred  with  confidence  level  2. 

Confidence  Level  1:  Certain 

2:  Reasonably  sure 
3:  Unsure 
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3.0  BASIS  FOR  TESTS 

Since  the  DM  was  not  forced  to  make  comparison  judgments,  but 
could  choose  to  be  indifferent  between  alternatives  or  choose  not  to 
answer  by  indicating  an  "unsure"  level  of  confidence  in  the  responses, 
the  forced  choice  model  which  used  equation  (2)  as  the  constraints 
was  inappropriate,  "unsure"  responses  were  dropoed  from  the  constraint 
set  and  t.>s  constraints  were  generated  by  the  following  inequality: 

l  (aik  -  ajk>  “k  i  «  <15> 

where  d  is  an  arbitrarily  small  positive  number 

If  all  responses  were  at  "unsure"  confidence  levels,  no  constraints 
can  be  generated  and  the  variables  must  be  obtained  using  a  modified 
version  of  LAYEVAL.  It  is  then  important  to  distinguish  between  two 
cases.  The  DM  indicating  "unsure"  confidence  levels  because  he/she,  i) 
did  not  want  to  make  a  preference  decision,  and  was  not  forced  to  make 
it,  or  ii)  was  sure  that  the  alternatives  were  very  similar,  and  hence 
was  indifferent  between  them.  In  the  first  case,  the  DM  has  not 
expressed  any  preference  at  all  hence  his  preferences  cannot  be 
quantified  by  means  of  determining  the  weights.  In  the  second  case, 
indifference  between  alternatives  A  and  B  can  generate  two  inequalities 
in  the  model  corresponding  to  A  preferred  to  B,  and  B  preferred  to  A, 
in  the  manner  described  by  Srinivasan  and  Shocker  [5,6].  For  this 
test,  the  distinction  between  the  above  two  cases  was  not  made, 
hence  the  "unsure"  responses  were  not  used  in  the  constraint  set. 

The  value  of  d  was  determined  from: 

M  x  d  =  1.0  (16) 

Where  M  =  number  of  ordered  pairs  for  which  j 
is  strictly  preferred  to  k 
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The  remaining  confidence  coefficients,  c.j,  associated  with 
"certain"  and  "reasonably  sure"  responses  were  utilized  in  several 
ways.  In  one  instance  they  were  set  at  the  arbitrary  values  2  and 
1  respectively.  In  this  instance,  it  was  felt  that  the  difference  in 
confidence  should  somehow  influence  the  determination  of  weights  so 
that  the  weights  would  favor  the  satisfaction  of  the  inequalities 
generated  by  the  "certain"  preferences  over  the  inequalities  generated 
by  the  "reasonably  sure"  preferences.  However,  the  choice  of  values 
for  the  confidence  coefficients  is  subjective  and  hence  not  defensible. 
Therefore,  for  the  second  instance,  another  set  of  results  was  obtained 
where  the  "certain"  and  "reasonably  sure"  responses  were  weighted 
equally. 

At  the  second  stage  of  the  model  it  was  noticed  that  the 
coefficients  associated  with  the  yilk  in  the  constraints,  using  the 
scaled  weights  of  stage  one,  were  generally  much  smaller  than  those 
associated  with  the  y^.  The  measurement  scale  for  the  flow 
attributes  was  therefore  increased  by  a  factor  of  20  (that  is,  the 
coefficients  of  y^^  were  divided  by  20)  in  order  to  have  coefficients 
of  about  the  same  scale  for  each  attribute.  This  scaling  was  done  only 
to  permit  a  better  visual  assessment  of  the  coefficients.  It  was  not 
required.  All  it  means,  in  operational  terms,  is  that  distances  should 
be  measured  in  units  of  20  basic  building  blocks  rather  than  in  units 
of  basic  building  blocks,  and  the  choice  of  distance  metric  could 
resolve  this. 

Also,  at  the  second  stage  of  the  model,  it  was  assumed  that  when 
the  DM  was  asked  to  indicate  in  which  of  two  layouts  he  preferred  the 
position  of  a  specific  activity,  considering  the  positions  of  other 
activities,  and  considering  all  attributes  simultaneously,  he  would 
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take  Into  account  only  those  attributes  for  which  he  had  been  able  to 
Indicate  a  definite  preference  at  the  individual  attribute  level.  For 
example,  if  a  definite  preference  was  expressed  considering  the 
attribute  noise,  and  a  definite  preference  was  expressed  considering 
the  temperature,  but  the  DM  could  not  express  a  preference  (or 
expressed  a  preference  at  the  "unsure"  confidence  level)  with  respect 
to  other  attributes,  then  when  the  DM  expressed  an  overall  preference. 

It  was  assumed  that  he/she  would  focus  his  attention  only  on  the 
attributes  noise  and  temperature  and  ignore  the  other  attributes. 

This  assumption  is  consistent  with  the  indication  by  the  DM  that 
he  tended  to  focus  more  on  the  very  different,  or  extreme  conditions 
and  Ignored  the  similar,  or  average  conditions.  In  the  preceding 
example  this  would  mean  that  the  constraint  generated  by  the  expressed 
overall  preference  at  the  second  stage  would  include  only  the  variables 
associated  with  the  noise  and  temperature  attributes.  It  is  again 
theoretically  possible  that  the  subscore  with  respect  to  a  specific 
attribute  is  at  the  "unsure"  confidence  level  for  every  constraint, 
thereby  implying  that  the  variables  associated  with  the  attribute  at 
the  second  stage  cannot  be  determined.  In  this  case,  although  the 
attribute  may  be  Important,  It  cannot  help  to  differentiate  among 
alternatives,  hence  It  Is  unimportant  for  prediction  purposes. 

An  assumption  conmonly  made  when  evaluating  layouts  is  that  costs, 
or  values,  vary  linearly  with  distance.  An  attempt  was  made  in  this 
case  study  to  determine  if  the  assumption  was  justified.  In  a  preliminary 
analysis,  the  responses  to  one  section  of  the  questionnaire  indicated 
that  the  DM's  values  varied  almost  linearly  with  distance.  Therefore, 
a  detailed  study  of  the  variations  of  value  with  distance  was  not 
pursued.  It  must  be  noted  that  the  analysis  was  restricted  to  the 
5  -  50  foot  distance  range.  It  is  conceivable  that  nonlinearities 
would  bd  noticed  if  wider  ranges  were  considered. 
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4.0  RESULTS 


The  numerical  results  from  the  optimization  process  are  reported 
in  two  major  categories  representing  the  experimental  options  on  the 
confidence  coefficients.  The  first  set  of  results  are  for  the  c.. 

*  J 

values  of  "certain"  and  "reasonably  sure"  set  arbitrarily  at  2  and  1 
respectively.  The  second  set  of  results  are  where  no  distinction  be¬ 
tween  "certain"  and  "reasonably  sure"  is  made,  i.e.  the  c^  values  are 
set  at  1  and  1. 

Within  each  c»t*s.  j  of  results,  two  stages  of  values  correspon¬ 
ding  to  the  staq*'?*  ,y(>  :t.mization  implied  by  (10)  and  (12)  for  stage  1 
and  (14)  for  staesfe  !  ••  'e  reported.  Thus,  one  could  view  the  stage  1 
results  as  related  v<  individual  attribute  levels  and  the  stage  2  re¬ 
sults  as  displaying  the  weights  between  attributes. 

Returning  to  the  CCM  methodology  described  in  section  1.1,  for 
two  layout  alternatives,  i  and  j,  an  expressed  preference  by  the  DM  for 
i  over  j  implies  that  s.  >  s..  Should  the  calculated  weights,  when 

*  J 

recombined  into  a  score,  actually  give  s.  _>  s-  this  would  be  considered 

*  J 

desirable.  However,  if  the  calculated  weights  give  si  >  s-  then  the 
estimated  weights  are  not  in  conformance  with  the  DM  paired  comparisons 
and  a  measure  of  goodness  of  fit  for  the  predicted  weights  can  be  de¬ 
veloped  as  described  below. 

Since  a  calculated  s.  greater  than  or  equal  to  s.  based  upon  the 

*  J 

weights  matches  the  DM's  preferences,  the  following  term  can  be  defined: 

07) 


<s,  -  r°,f 

l<5j  -  si>  ff  sj  >  si 


or 


(si  -  Sj)‘  *  max  [0,  (s^  -  s^] 


X 


( 
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Thus  (s.  -  s.)"  is  a  measure  of  error  corresponding  to  the  layouts  i 

and  j  and  a  given  solution  to  the  weights.  Considering  all  of  the 

layouts  in  ft  (from  (2)),  define  the  following: 

b  =  i  (s,  -  Sjr 
(i,j)  £  ft 

as  a  measure  of  poorness  of  fit  for  the  calculated  weights  over  the 
set  of  layouts,  ft,  for  which  preference  data  was  expressed.  It  is 
desirable  to  minimize  B. 

Unfortunately,  B  can  be  minimized  in  a  trivial  fashion  by  simply 
having  all  weights  equal  zero.  Then  s.  =  s.  =  0,  still  satisfying 

'  J 

s.  _>  s..  This  type  of  solution  yields  no  useful  information  and  a 
modified  approach  to  the  problem  must  be  taken.  Inequality  (15)  con¬ 
sidered  a  small  positive  difference,  d,  for  the  attributes'  score  for 
layouts  i  and  j.  This  required  difference  over  all  layouts  in  ft  can 
be  defined  as  h  where: 


(19) 


£  (s,  -  si)  =  h 

(i,j)  e  n  3 


(20) 


and  h  is  an  arbitrary  positive  constant  that  merely  serves  as  a  scaling 
factor.  A  new  index  of  fit  can  now  be  defined  as: 


C  = 


B 


(21) 


h  +  B 

where  C  is  bounded  by  0  and  1.  Now  any  solution,  other  than  the  trivial 
solution  of  zero  weights,  that  optimized  B  will  also  optimize  C.  It  is 
therefore  desirable  to  minimize  C  and  the  C  values  are  displayed  in  all 


results. 
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4.1  RESULTS  FOR  UNEQUAL  CONFIDENCE  COEFFICIENTS 

It  should  be  recalled  that  the  confidence  coefficients  for  the 
following  results  were  arbitrarily  set  at  a  value  of  2  for  "certain"' 
and  a  value  of  1  for  "reasonably  sure"  in  analyses  based  on  the  compari¬ 
sons  shown  in  Table  I. 

4.1.1  STAGE  1  RESULTS 

The  calculated  weights  for  the  attribute  noise  appear  in  Table 


III. 


(2)  (n+m)  means  that  n  were  expressed  at  the  "certain"  level  of  confidence,  and  m  were  expressed  at  the 
"reasonably  sure"  level  of  confidence. 
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Considering  the  scaled  weights  in  the  first  block  associated  with 
W  U  ^  anc*  W  it  is  noticed  that  the  weights  increase  with  m. 

This  appears  reasonable  since,  in  general,  lower  noise  levels  are  pre¬ 
ferred  (and  scores  were  defined  so  that  lower  weights  indicate  lower 
penalties).  The  same  situation  is  noticed  with  respect  to  W  W  ^ 
and  W  -jg.  The  weights  associated  with  W  W  lg  and  W  ig  are,  however, 
almost  the  same,  indicating  that  they  would,  in  general,  not  be  very 
helpful  in  predicting  preferences.  The  DM  agreed  with  this  interpreta¬ 
tion  at  the  second  interaction  session  with  him,  since  in  his  opinion, 
if  an  activity  had  a  high  noise  level,  the  noise  level  of  adjacent  ac¬ 
tivities  could  do  little  to  improve  or  worsen  the  situation  anyway. 

The  index  of  fit  is  good  for  the  first  three  and  the  last  three  but 
not  very  good  for  the  middle  three.  One  reason  for  the  poorer  fit,  in¬ 
dicated  by  the  DM,  was  that  he  tended  to  focus  more  on  quiet  activities 
to  ensure  that  their  performance  was  not  affected  by  noisy  neighbors, 
and  on  noisy  activities  to  check  which  activities  around  them  were  being 
affected  by  the  noise.  The  DM  tended  to  ignore  activities  at  the  medium 
noise  levels  since  their  levels  could  be  more  easily  controlled  (by 
soundproofing)  if  this  was  desired,  at  the  final  ship  design  stage. 
Another  reason  is  that  although  the  index  of  fit  as  defined  in  (21),  is 
claimed  by  Srinivasan  and  Shocker  to  be  largely  independent  of  the  number 
of  constraints  in  the  constraint  set,  it  is  likely  that  better  fits  are 
obtained  with  fewer  constraints  at  the  expense  of  providing  worse  e  ti- 
mates  of  the  weights. 

Similar  inferences  for  the  attributes  temperature  and  humidity  may 
by  drawn  from  the  results  presented  in  Tables  IV  and  V. 


Results  for  the  attribute  Temperature  with  confidence  coefficients  of  2  and  1  (stage  1) 
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(1)  scaled  so  that  the  weights  of  the  separate  subproblems  sum  to  one. 

(2)  (n+m)  means  that  n  were  expressed  at  the  "certain"  level  of 

confidence,  and  m  were  expressed  at  the  "reasonably  sure"  level  of  confidence. 
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Table  V 


Results  for  the  attribute  Humidity  with 
confidence  coefficients  of  2  and  1  (stage  1) 


Unknown  Variables 

W  31  W  32 

W  33 

W  34 

Numerical  Weights 

0.  .966 

0. 

1.562 

Scaled  WEights  (1) 

0  1.0 

0. 

1. 

Objective  function  value 

156.9 

94.1 

Index  of  fit,  C 

.61 

.48 

Number  of  correctly  predicted 
preferences  (2) 

33(21+12) 

20(12+8) 

Number  of  wrongly  predicted 
preferences  (2) 

18(12+6) 

7(5+2) 

Overall  number  of  preferences 

corrently  predicted  by  weights  (2)  53(33+20) 

Overall  number  of  preferences 

wrongly  predicted  by  weights  (2)  25(17+8) 

Overall  number  of  preferences 
indicated  (at  other  than  the 

unsure  confidence  level)  (2)  78(50+28) 


(1)  scaled  so  that  the  weights  of  the  separate  subproblems  sum  to  one. 

(2)  (n+m)  means  that  n  were  expressed  at  the  "certain"  level  of 
confidence,  and  m  were  expressed  at  the  "reasonably  sure"  level  of 
confidence. 


In  Table  V,  the  weights  indicate  that  the  lower  humidity  level  is 
preferred.  The  DM  agreed  with  this  interpretation.  The  indices  of  fit 
were  not  as  good  as  those  for  noise  and  temperature,  possibly  due  to  the 
fewer  levels  used  for  this  attribute.  The  two  levels  result  in  four 
variables  used  to  account  for  the  preferences  indicated  by  the  DM.  For 
noise  and  temperature,  the  three  levels  allowed  for  nine  variables. 


IDSPUiWIII  )■  I"  i 
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4.1.2  STAGE  2  RESULTS 

The  second  stage  results  reported  in  Table  VI  were  also  scaled 
in  the  manner  described  in  the  preceding  section  for  the  first  stage. 
These  data  were  scaled  so  that  the  variables  summed  to  thirty. 

Although  the  scaling  is  not  required  technically, the  arbitrary  value 
to  thirty  was  chosen  to  permit  a  better  visual  assessment. 

.  In  Table  VI,  it  is  instructive  to  consider  the  variables 

^  associated  with  the  first  three  data  columns  separately  from  those 

associated  with  the  last  three  data  columns  for  intrinsic  attributes 
and  the  first  data  column  separate  from  the  latter  data  columns  for 
!  flow  attributes.  As  a  group  the  former  are  associated  with  low  levels 

I  of  functional  (or  flow  attribute)  connection  between  activities,  and 

i 

cases  where  it  is  less  important  that  activities  have  compatible 
neighbors  (in  the  sense  of  intrinsic  attribute  levels).  It  is 
observed  that  only  three  out  of  the  ten  variables  in  this  first  group 
are  at  positive  levels.  Obtaining  zero  weights  for  these  variables 
appears  reasonable. 
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Table  VI 

Second  stage  results  with  confidence 
coefficients  of  2  and  1  and  d  =  .00124 


Noise 


Unknown  variables 

ylll 

y121 

y131 

Y21 1 

y221 

y231 

Numerical  Weights 

0. 

0. 

0. 

,02867 

.00043 

.0 

Scaled  Weights 

0. 

0. 

0. 

.44 

.007 

.0 

Temperature 

Unknown  variables 

yl  12 

yl  22 

y  1 32 

y212 

y222 

y232 

Numerical  weights 

.02054 

.00043 

.0 

.04462 

.00393 

.0 

Scaled  weights 

.318 

.007 

.0 

.69 

.061 

.0 

Humidity 

Unknown  variables 

yl  1 3 

yl  23 

y213 

y223 

Numerical  weights 

.0 

.00225 

.0025 

1.45142 

Scaled  weights 

.0 

.035 

.039 

22.447 
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Nearness  to  desired  deck 


Unknown  variables 

y114 

y214 

Numerical  weights 

.0 

.33377 

Scaled  weights 

.0 

5.162 

Functional  Performance 

Unknown  variables 

y115 

y215  y3I5 

Numerical  weights 

.0 

.0  .05127 

Scaled  weights 

.0 

.0  .793 

Objective  function  value 

.91 

Index  of  fit,  C 

.01 

Overall  number  of  preferences 

66(43+23) 

correctly  predicted  by  weights  (2) 

Overall  number  of  preferences 

21(12+9) 

wrongly  predicted  by  weights  (2) 

Overall  number  of  preferences 

87(55+32) 

Indicated  (at  other  than 

the  unsure  confidence  level)  (2) 

(1)  scaled  so  that  the  weights  sum  to  30 

(2)  (n+m)  means  that  n  were  expressed  at  the  "certain"  level  of 
confidence,  and  m  were  expressed  at  the  "reasonably  sure"  level 
of  confidence. 
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since  the  zero  weights  imply  no  penalty  for  adverse  conditions  when  the 
DM  has  indicated  that  in  the  particular  situation  the  conditions  are 
only  moderately  important  or  not  important.  Also  note  that  the  three 
positive  variables  are  numerically  smaller  than  their  counterparts  in  the 
second  group  that  are  greater  than  zero.  This  is  also  a  reasonable 
result  since  higher  penalties  should  be  assigned  to  adverse  conditions 
when  the  DM  has  indicated  that  it  is  important  to  have  good  conditions. 

Consider  the  variables  associated  with  the  second  group  that  have 
values  greater  than  one.  These  correspond  to  situations  in  which  it 
has  been  indicated  that  adverse  conditions  should  be  penalized.  It 
is  observed  that  only  three  of  the  eleven  variables  are  at  zero  levels. 
This  is  an  encouraging  result  considering  that  two  of  them,  and 
y232’  were  exPected  t0  be  zer0  ™  accordance  with  the  DM's  preference 
structure.  These  two  variables  are  associated  with  activities  which 
are  at  high  levels  on  the  attributes  noise  and  temperature.  It  was 
indicated  by  the  DM  that  it  matters  little  in  these  cases  what  the 
neighboring  activities  are.  The  problem  of  high  noise  and  temperature 
levels  must  be  solved  by  controlling  the  interval  characteristics  of  the 
activities  rather  than  trying  to  reduce  the  levels  by  surrounding  them 
with  activities  at  low  noise  and  temperature  levels. 

The  zero  weight  assigned  to  yg^g  was  considered  extremely  unusual 
by  the  DM.  The  DM  considered  the  second  flow  attribute  to  be  very 
important  in  determining  the  overall  preferences  for  the  activities, 
whereas  the  zero  weight  meant  that  the  second  level  of  this  important 
attribute  was  not  being  used  by  the  model  to  predict  preferences. 


In  order  to  reconcile  this  discrepancy,  checks  were  made  at  the 
individual  attribute  level.  At  the  first  session,  the  DM  had  been  asked 
to  predict  preferences  between  activities  in  paired  layouts  while 
considering  only  the  second  flow  attribute  (this  set  of  preferences 
was  not  necessary  to  determine  the  weights).  It  was  noticed  that  the 
weights  obtained  did  not  predict  these  preferences  well.  An  examination 
of  the  raw  data  for  the  expressed  preferences  revealed  that,  in  some 
cases,  dominated  alternatives  were  preferred  by  the  DM.  Therefore  at 
the  second  session,  the  DM  was  asked  to  explain  his  reasoning  for  the 
selection  of  the  preferred  alternatives,  while  considering  only  the 
second  flow  attribute. 

It  was  then  discovered  that  a  major  factor,  related  to  the  social 
structure  within  a  ship,  had  not  been  considered.  The  DM  indicated 
that  the  revised  input  would  result  in  a  table  "with  a  liberal  use 
of  negative  entries."  The  crew,  the  officers,  and  the  captain 
constituted  three  social  levels  within  a  ship,  making  their  locations 
on  separate  decks  desirable,  especially  with  respect  to  eating,  living 
and  recreation  areas.  The  desire  was,in  some  cases }strong  enough  to 
constitute  a  requirement,  meaning  that  a  layout  design  could  be  rejected 
based  on  a  single  unacceptable  situation.  This  "social  structure" 
attribute  was  not  expressly  included  by  the  Navy  ship  design  personnel 
as  an  appropriate  attribute  and  hence  not  discussed  with  the  DM  when 
preferences  were  elicited  from  him,  It  was  also  not  considered  when 
designing  the  four  layouts  used  for  preference  judgments.  This 
resulted  in  some  infeasible  layouts  (considering  this  attribute),  thus 
violating  an  important  assumption  of  the  model  that  all  layouts  must  at 
least  be  at  an  acceptable  level  with  respect  to  requirements  (or  be 
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feasible)  before  preference  questions  are  asked  to  measure  values 
for  attibutes. 

Such  violated  requirements  have  a  serious  impact  on  the  ability 
of  the  model  to  correctly  predict  preferences  for  entire  layouts.  A 
layout  could  be  much  better  than  another  in  every  way  but  could  be 
rejected  (and  hence  not  preferred  over  the  other)  based  on  only  one 
violated  requirement. 

The  violation  of  the  requirements  does  not  however  invalidate  the 
conclusions  drawn  earlier.  As  long  as  the  violations  are  more  the 
exception  rather  than  the  rule,  the  numerical  values  assigned  to  the 
variables  should  remain  reasonable.  But  the  violations  will  tend  to 
increase  the  mismatch  between  preferences  predicted  by  the  model  and 
the  preferences  expressed  by  the  DM  since  it  will  in  general  be 
difficult  to  satisfy  the  inequalities  generated  by  the  preferences  which 
result  from  requirement  violations.  Also,  the  burden  of  "explaining" 
the  preferences  due  to  the  omitted  attribute  will  fall  on  the  included 
attributes,  thereby  tending  to  move  the  estimates  of  the  variables 
associated  with  the  included  attributes  away  from  their  "true  values." 

There  is  a  tendency  for  the  model  to  "weigh  more,"  in  the  sense  of 
better  estimate,  the  variables  associated  with  the  columns  which  are 
dense.  The  numerical  values  of  the  variables  associated  with  the 
sparse  columns  would  be  more  sensitive  to  changes  in  parameters.  For 
this  small  case  study  some  attribute  levels  were  dropped  from  con¬ 
sideration,  since  not  enough  inequalities  of  the  form  in  (15)  could 
be  generated  to  obtain  the  numerical  weights  of  the  associated 
variables  with  any  degree  of  confidence.  The  preference  questions 
were  asked  about  all  activities  in  order  to  obtain  the  weights.  There¬ 
fore  the  sparseness  of  the  columns  associated  with  the  variables  could 
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not  be  controlled.  It  was  a  function  of  how  the  activities  were 
classified  along  attributes.  Hence  some  variables  were  "weighted" 
more  than  others  simply  because  the  variables  appeared  more  frequently 
and  hence  their  columns  were  denser.  When  studies  are  conducted  for 
cases  which  have  many  more  activities  (i.e,,  large  ship  designs),  then 
it  may  be  possible  to  select  only  a  subset  of  activities  about  which 
preference  questions  are  asked  to  generate  constraints,  and  this 
selection  could  be  made  such  that  the  resulting  columns  associated 
with  the  variables  are  (approximately)  equally  dense. 

One  last  observation  reguarding  the  weights  in  Table  VI  is  that 
the  scaled  numerical  values  of  y22g  an(*  y2-|4  are  "unusua^y  high." 

Note  that  judgments  regarding  the  significance  of  the  attributes  should 
not  be  made  from  the  numerical  values  associated  with  the  variables. 
These  values  get  affected  by  the  scales  of  measurement  used  for  the 
attributes.  The  comments  regarding  the  weights  should  be,  and  have 
been  restricted  to  comparisons  of  weights  within  attributes.  The  only 
absolute  value  about  which  comments  have  been  made,  have  been  the  zero 
numerical  values  associated  with  some  of  the  variables,  implying  that 
the  variables  are  insignificant, 

One  possible  explanation  for  the  "high"  value  of  y214  is  that  in 
this  test,  it  was  noticed  at  the  second  stage  that  the  coefficients 
associated  with  the  intrinsic  attribute,  using  the  scaled  weights  of 
stage  one,  were  generally  much  smaller  than  those  associated  with  the 
flow  attributes.  The  measurement  scales  for  the  flow  attributes  were 
therefore  increased  by  a  factor  of  20  in  order  to  have  coefficients 
of  about  the  same  scale  for  each  attribute.  Had  this  not  been  done,  the 
numerical  value  of  y2^4  would  have  been  5.152/20  *  .2581  instead. 
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1  This  scaling  was  done  only  for  easier  visual  assessment  of  the 

coefficients.  It  was  not  necessary  andjas  mentioned  earlier,  it  does 
not  reflect  the  significance  of  the  attributes. 

There  is  however  a  different  factor  ..jssibly  responsible  for  the 
high  numerical  value  assigned  to  y^.  Consider  the  column  of  a 
(  variable  in  the  constraint  matrix  generated  by  (15).  If  the  column 

has  only  one  non-zero  entry  and  this  entry  is  negative  (say),  then 
|  for  the  case  where  the  inequality  associated  with  the  row  of  that  entry 

is  not  satisfied  by  the  assigned  weights,  a  unit  increase  in  the  variable 

i 

'  will  increase  the  objective  by  the  coefficient  of  the  negative  entry. 

If  there  were  ten  such  entries,  then  the  objective  would  increase  by 
the  sum  of  their  coefficients.  There  is  therefore  a  tendency  for  the 
model  to  "weigh  more,"  in  the  sense  of  better  estimate,  the  variables 
associated  with  the  columns  which  are  dense.  The  numerical  values  of 
the  variables  associated  with  the  sparse  columns  would  be  more  sensitive 
to  changes  in  parameters.  There  is  only  one  activity,  activity  18, 

1 

j  associated  with  y22^  ancl  hence  its  column  is  sparse.  This  could  have 

J  resulted  in  greater  instability,  and  hence  the  "high"  value.  The 

ij  argument  might  be  used  for  dropping  some  attribute  levels  from 

*1  consideration,  since  not  enough  inequalities  of  the  form  in  (15) 


could  be  generated  to  obtain  the  numerical  weights  of  the  associated 
variables  with  any  degree  of  confidence. 

Examining  parametrically  over  values  of  d  (see  Tables  VII  and 
VIII,  it  was  noticed  that  the  results  are  quite  insensitive  to  the 
choice  of  "d."  Increasing  d  parametrically  by  more  than  2000%  over 
the  recommended  value  did  not  affect  the  number  of  correctly  predicted 
preferences.  Based  on  this  limited  experimentation,  it  therefore  appears 
that  the  predictive  power  of  the  model  is  quite  insensitive  to  the  choice 
of  d. 
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Table  VII 


Unknown  variables 

Second  stage  results  with  confidence 

coefficients  of  2  and  1  and  d  *  .05 

Noise 

ylll  yl 21  y131  y211 

y222 

y231 

Numerical  Weights 

.0 

.0 

.0 

.12743 

.00191 

.0 

Scaled  Weights 

.0 

.0 

.0 

1.053 

.016 

.0 

Temperature 

Unknown  variables 

yl  1 2 

yl  22 

yl  32 

yZ12 

y222 

y232 

Numerical  weights 

.09138 

.00191 

.0 

.19848 

.01748 

.0 

Scaled  weights 

.755 

.015 

.0 

1.64 

.144 

.0 

Humi di ty 

Unknown  variables 

yl  1 3 

y  1 23 

y213 

y223 

Numerical  weights 

.0 

.01 

.01112 

1 .45882 

Scaled  weights 

.0 

.083 

.092 

12.052 
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t. 


Nearness  to  desired  deck 


Unknown  variables 

yl  1 4 

y214 

Numerical  weights 

.0 

1.48475 

Scaled  weights 

.0 

12.266 

Functional  Performance 

Unknown  variables 

y115 

y215 

y315 

Numerical  weights 

.0 

.0 

.22809 

Scaled  weights 

.0 

.0 

1.884 

Objective  function  value 

Index  of  fit,  C 

Overall  number  of  preferences 
correctly  predicted  by  weights  (2) 

Overall  number  of  preferences 
wrongly  predicted  by  weights  (2) 

Overall  number  of  preferences 
indicated  (at  other  than 
the  unsure  confidence  level)  (2) 

(1)  scaled  so  that  the  weights  sum  to  30 

(2)  (n+m)  means  that  n  were  expressed  at  the  "certain"  level  of 
confidence,  and  m  were  expressed  at  the  "reasonably  sure"  level 
of  confidence. 


4.05 

.0 

66(43+23 

21(12+9 

87(55+32 


Table  VII 


Second  stage  results  with  confidence 


coefficients  of  2  and 

1  and  d  = 

.25 

Noise 

Unknown  variables 

ym 

yl  21 

yl  31 

y21 1 

y222 

y  231 

Numerical  Weights 

.0 

.0 

.0 

7.47599 

.00953 

.0 

Scaled  Weights 

.0 

.0 

.0 

12.112 

.015 

.0 

Temperature 

Unknown  variables 

y112 

y122 

yl  32 

y212 

y222 

y231 

Numerical  weights 

.0 

.00954 

.0 

.99238 

.08742 

.0 

Scaled  weights 

.0 

.15 

.0 

1.608 

.42 

.0 

Humidity 

Unknown  variables 

y113 

y123 

y213 

y223 

Numerical  weights 

.0 

.05 

.05561 

1.29871 

Scaled  weights 

.0 

.081 

.09 

2.104 

,r 


Nearness  to  desired  deck 


Unknown  variables 

y114 

V21 4 

Numerical  weights 

.0 

7.42376 

Scaled  weights 

.0 

12.027 

Functional  Performance 


Unknown  variables 

y115 

y215 

•y31 5 

Numerical  weights 

.0 

,0 

1.14043 

Scaled  weights 

.0 

.0 

1.848 

Objective  function  value  20.24 

Index  of  fit,  C  .17 

Overall  number  of  preferences  66(43+23) 

correctly  predicted  by  weights  (2) 

Overall  number  of  preferences  21(12+9) 

wrongly  predicted  by  weights  (2) 

Overall  number  of  preferences  87(55+32) 

indicated  (at  other  than 

the  unsure  confidence  level)  (2) 

(1)  scaled  so  that  the  weights  sum  to  30 

(2)  (n+m)  means  that  n  were  expressed  at  the  "certain"  level  of 
confidence,  and  m  were  expressed  at  the  "reasonably  sure"  level  of 
confidence. 


4.2  RESULTS  FOR  EQUAL  CONFIDENCE  COEFFICIENTS 

For  these  data,  the  confidence  coefficients  were  arbitrarily  set 
at  1  for  "certain"  and  1  for  "reasonably  sure."  Hence  both  degrees 
of  certainty  carried  equal  weight  within  the  mathematical  model. 

4.2.1  STAGE  1  RESULTS 

The  calculated  weights  for  the  intrinsic  attributes  of  Noise, 
Temperature  and  Humidity  appear  in  Tables  VIII,  IX  and  X  respectively. 


Results  for  the  attribute  Temperature  with 
confidence  coefficients  of  1  and  1  (stage  1) 
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Table  X 

Results  for  the  attribute  Humidity  with 
confidence  coefficients  of  1  and  1  (stage  1) 


Unknown  Variables 

w  31 

w  32 

W  33 

w  34 

Numerical  Weights 

.0 

2.019 

.0 

2.1Q- 

Scaled  Weights  (1) 

.0 

1.0 

.0 

1.0 

Objective  function  value 

190.17 

74.84 

Index  of  fit,  C 

.66 

.43 

Number  of  correctly  predicted 
preferences  (2) 

33(21+12) 

20(12+8) 

Number  of  wrongly  predicted 
preferences  (2) 

18(12+6) 

7(5+2) 

Overall  number  of  preferences  53(33+20) 

correctly  predicted  by  weights  (2) 

Overall  number  of  preferences  25(17+8) 

wrongly  predicted  by  weights  (2) 

Overall  number  of  preferences  78(50+28) 

indicated  (at  other  than  the 
unsure  confidence  level)  (2) 

(1)  scaled  so  that  the  weights  of  the  separate  subproblems  sum  to  one 

(2)  (n+m)  means  that  n  were  expressed  at  the  "certain"  level  of 
confidence,  and  m  were  expressed  at  the  "reasonably  sure"  level 
of  confidence. 
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These  data  are  quite  similar  to  those  of  Tables  III,  IV,  and  V  and 
much  of  the  earlier  discussion  continues  to  apply  here.  The 
slightly  better  predictive  performance  in  the  mid-range  levels  of 
the  attributes  is  not  considered  significant. 
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4.2.2  STAGE  2  RESULTS 

The  second  stage  results  are  reported  in  Table  XI.  These  data 
were  scaled  so  as  to  have  the  variables  sura  to  30  for  the  simple  expedient 
of  improved  visual  assessment.  The  differences  between  the  values 
in  Table  XI  and  Table  VI  are  considered  slight  and  are  discussed  below. 

The  numerical  value  assigned  to  yg-j s  at  the  second  stage  is 
positive  instead  of  zero.  Although  this  is  the  desired  result,  the 
result  does  not  negate  the  discussion  in  an  earlier  section  regarding 
the  social  structure  attribute.  The  fact  remains  that  the  DM  con¬ 
sidered  the  attribute  important  and  it  should  be  explicitly  included  in 
the  model.  A  possible  reason  for  the  nonzero  value  is  that  with 
confidence  coefficients  of  1  and  1,  the  "certain"  responses,  and  hence 
the  preferences  due  to  the  requirement  considerations,  are  weighted 
less  than  they  are  with  confidence  coefficients  of  2  and  1.  Therefore 
requirement  violations  which  result  in  mismatches  tend  to  remain 
hidden  because  they  do  not  influence  the  determination  of  the 
numerical  weights  any  more  than  the  "reasonably  sure"  responses  do. 

This  observation  may  also  explain  the  second  significant  difference 
in  results. 

This  second  significant  difference  for  the  derived  scores  is 
that  the  model  does  not  predict  overall  preferences  any  better  when 
only  "certain"  subcomponents  of  score  are  used  to  obtain  the  overall 
score.  This  may  likely  be  due  to  the  fact  that  since  the  overall 
perferences  expressed  by  the  DM  were  often  Influenced  heavily  by 
violated  requirements,  the  model  with  confidence  coefficients  of  2 
and  1  predicts  overall  preferences  better  using  "certain"  subcomponents 
since  It  "weighs"  the  preferences  due  to  requirement  consideration 
more  when  determining  the  weights,  The  model  with  confidence 
coefficients  of  2  and  1  does  not. 
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Table  XI 

Second  stage  results  with  confidence 
coefficients  of  1  and  1  and  d  =  .00124 


Noise 


Unknown  variables 

ylll 

yl  21 

yl  31 

y211 

y222 

y231 

Numerical  Weights 

.0 

.0 

.0 

.02841 

.0 

.0 

Scales  Weights 

.0 

.0 

.0 

.26984 

.0 

.0 

Temperature 

Unknown  variables 

y112 

yl  22 

yl  32 

y212 

y222 

y232 

Numerical  weights 

.03343 

.0 

.0 

.04539 

.00495 

.0 

Scaled  weights 

.31748 

.0 

.0 

.43112 

.43112 

.0 

Humi di ty 

Unknown  variables 

yl  13 

yl  23 

y213 

y223 

Numerical  weights 

.0 

.00225 

.00187 

2.63527 

Scaled  weights 

.0 

.02135 

.01779 

25.01844 

j± 
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Nearness  to  desired  deck 


Unknown  variables 

y114 

y214 

Numerical  weights 

.0 

.34606 

Scaled  weights 

.0 

Functional  Performance 

3.28659 

Unknown  variables 

y115  y215 

y31 5 

Numerical  weights 

.0  .00899 

.05317 

Scaled  weights 

.0  .08540 

.50494 

Objective  function  value 


.54 


Index  of  fit,  C 


.01 


Overall  number  of  preferences 
correctly  predicted  by  weights  (2) 


69(44+25) 


Overall  number  of  preferences  18(11+7) 

wrongly  predicted  by  weights  (2) 

Overall  number  of  preferences  87(55+32) 

indicated  (at  other  than 

the  unsure  confidence  level)  (2) 


(1)  scaled  so  that  the  weights  sum  to  30 


(2)  (n+m)  means  that  n  were  expressed  at  the  "certain"  level  of 

confidence,  and  m  were  expressed  at  the  "reasonably  sure"  level 
of  confidence. 


>!  .  • 


—2=23 
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5.0  LAYOUT  SCORES 

In  the  questionnaire  administered  to  the  DM,  he  had  been  asked 
to  indicate  his  preferences  in  the  matrix  which  is  represented  in 
Block  form  in  Figure  II. 


Individual  Attribues  All  Attributes 


Individual 

Activities 


Entire  Layout 


Figure  II.  Black  Diagram  of  Preference  Responses 


The  responses  in  Block  A,  corresponding  to  individual  activities 
and  individual  attributes,  were  used  to  obtain  the  values  of  the 
variables  of  the  first  stage  of  the  model.  The  reponses  in  Block  B, 
corresponding  to  individual  activities  and  all  attributes  considered 
simultaneously,  were  used  to  obtain  the  values  of  the  variables  of 
the  second  stage  of  the  model . 

Using  the  weights  obtained  in  the  two  stages,  it  is  possible, 
according  to  the  proposed  aditive  model,  to  predict  the  preferences 
of  DM  for  entire  layouts.  The  DM  had  been  asked  at  the  first  session 
to  indicate  his  preferences  for  layouts  considering  each  attribute 
separately  (Block  C)  followed  by  considering  all  the  attributes 
simultaneously  (Block  D).  An  attempt  was  therefore  made  to  see  if 
the  preferences  predicted  by  the  model  for  Blocks  C  and  D  matched 
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the  preferences  indicated  by  the  DM. 

The  DM's  responses  for  the  cases  when  he  was  not  unsure  of  his 
responses,  and  the  predictions  of  the  model  are  indicated  in  Table  XII 
where  the  confidence  coefficients  were  2  and  1.  The  predictions  of 
the  model  for  Block  C  (considering  each  attribute  separately)  were 
the  same,  irrespective  of  whether  the  weights  corresponding  to  d= 
.00124,  d=.05,  or  d=.25  were  used.  This  indicates,  from  the  limited 
sensitivity  test,  that  the  predictive  power  of  the  model  is  quite 
insensitive  to  the  choice  of  d.  Considering  all  the  attributes 
simultaneously  (Block  D),  the  results  with  d=.25  predicted  one  extra 
pairwise  comparison  correctly,  which  the  others  did  not.  Hence  this 
set  of  weights  was  used  for  the  predictions  in  Table  XII. 

Note  that  the  predictions  and  scores  considering  all  the  defined 

* 

attributes  simultaneously  (Block  D)  are  dependent  on  the  predictions 
and  scores  considering  the  attributes  separately  (Block  C).  Hence 
when  considering  the  number  of  correctly  and  wrongly  predicted 
preferences,  one  should  ideally  not  combine  the  results  of  Blocks 
C  and  D, 

The  validation  of  this  stage  of  the  model  was  made  difficult 
because  of  the  missing  input  on  flow  attributes,  due  to  the  DM 
incorporating  social  requirements  in  his  responses.  Although  location 
restrictions  due  to  social  structures  were  considered  by  the  DM 
when  expressing  some  preferences,  these  considerations  were  not 
originally  delineated.  Thus,  some  of  these  considerations  constituted 
requirements  of  a  constraining  type  discussed  earlier,  and  these 
constraints  were  violated  by  the  proposed  layout  designs. 
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Table  XII 

Comparison  of  actual  preferences  for  the 
entire  layout  versus  predicted  preferences 
using  confidence  coefficients  of  2  and  1. 


Block  C 


Attribute 

Layout  Confidence  Layout  Prediction  Prediction  of 

Pair  (1)  Level  (2)  preferred  of  model  (3)  model  consid- 

by  DM  ering  "certain" 

subscores  (4) 

Noi  se 

AB 

1 

B 

-1 

-1 

BC 

1 

e 

1 

1 

CD 

1 

C 

1 

1 

AC 

1 

c 

1 

1 

AD 

1 

D 

-1 

-1 

BD 

2 

D 

1 

1 

Temperature 

CD 

1 

D 

1 

1 

AD 

1 

D 

-1 

1 

BD 

1 

D 

1 

1 

Humidity 

CD 

1 

D 

-1 

1 

AD 

1 

D 

-1 

— 

BD 

1 

D 

-1 

Functional 

Performance 


Attributes 

considered 

simul¬ 

taneously 


Number  of  correct  and  wrong 
predictions  considering  attributes 
simultaneously  (Block  D) 

Number  of  correct  and  wrong 
predictions  considering  attributes 
separately  (Block  C) 


+  2,  -2 


+  10,  -7 


+  2,  -2 


+  12,  - 


(1)  A,  B,  C,  D  were  four  layouts  used  for  pairwise  comparisons 

(2)  1:  certain  2:  reasonably  sure 


(3)  1:  correct  prediction 


-1 :  wrong  prediction 


(4)  same  as  (3). 


indicates  prediction  was  not  possible 


w 
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Consequently  there  were  layouts  considered  to  be  very  good  in  most 
respects,  but  they  were  rejected  based  on  certain  unsatisfied 
requirements. 


f 


l 


u 
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Some  requirements,  other  than  those  related  to  the  social  attribute, 
were  also  uncovered  when  the  DM  was  asked  to  give  the  reasoning  behind 
certain  choices  that  were  made,  For  example  the  DM  considered  it  very 
undesirable  to  have  the  machinery  area  near  the  freezer  area  because 
of  temperature  effects.  It  was  extremely  undesirable  to  have  the 
engine  room  below  the  medical  wardroom  (the  situation  was  encountered 
in  Layout  A)  because  of  the  noise  generated  by  the  engine  room. 

Were  Layout  A  not  penalized  so  much  by  the  DM  on  basis  of  the 
one  violated  requirement,  the  performance  of  the  model  would  have 
improved  so  that  out  of  the  seven  wrongly  predicted  preferences  in 
Block  C,  five  could  have  been  correctly  predicted,  resulting  in  a  net 
score  of  15  correct  predictions  and  2  incorrect.  Similarly  in  Block  D. 
all  four  preferences  could  have  been  correctly  predicted.  It  should 
be  "Oted  that  this  is  merely  speculation  since  other  requirements  were 
also  involved.  The  important  point  is  that  such  requirement  violation 
problems  should  not  exist  in  layouts  used  for  comparison  judgments. 

Such  instances  represent  cases  where  the  DM  is  uncompromising.  The 
model  is  however  concerned  with  obtaining  weights  for  evaluation  where 
such  rigid  standards  have  been  met,  and  a  choice  must  be  made  con¬ 
sidering  the  attributes,  and  the  trade-offs  and  compromises  that  the 
DM  is  willing  to  make  among  them. 

In  an  attempt  to  focus  on  the  requirement  aspects  in  order  to 
make  predictions  (something  that  the  model  was  not  originally  designed 
to  do),  scores  were  computed  once  again  for  Block  C  by  considering  only 
those  subcomponents  of  total  scores  where  the  DM  had  made  a  preference 
judgment  in  Block  A  at  the  "certain"  level  of  confidence  when  he  was 
compa'  ng  layouts  with  respect  to  individual  activities  and  attributes, 

The  assumption  here  is  that  if  any  requirement  was  violated,  the  DM  was 
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likely  to  be  "certain"  of  his  choice,  Thus  if  the  DM  indicated  that  he 
preferred  Layout  A  over  Layout  B  at  the  "certain"  confidence  level 
when  considering  activity  2  with  respect  to  the  noise  attribute,  then  the 
sco*  of  activity  2  with  respect  to  noise  would  be  considered  when 
predicting  the  choice  between  Layouts  A  and  B.  If  the  confidence 
level  was  any  less  than  "certain"  this  part  of  the  score  would  not 
be  considered. 

Predictions  made  on  this  assumption  resulted  in  a  much  better 
predictive  capability  of  the  model  for  Block  C.  There  were  12  correct 
predictions  and  3  wrong  ones.  Two  predictions  could  not  be  made  because 
of  intransitivities.  These  intransitivities  may  be  expected  because 
subcomponents  of  score  may  be  considered  in  one  instance  and  not  in 
others.  As  an  example,  consider  the  case  where  by  using  the  subcomponents 
of  the  score.  Layout  A  is  preferred  overall  to  Layout  B  which  is  preferred 
to  Layout  C.  But  Layout  A  could  be  predicted  to  be  indifferent  with 
respect  to  Layout  C  if  all  preferences  between  A  and  C  were  at  less  than 
"certain"  levels  of  confidence. 

In  general,  the  results  using  confidence  coefficients  of  1  and  1 
as  presented  in  Table  XIII  are  very  similar  to  the  results  using 
confidence  coefficients  of  2  and  1.  The  numerical  values  of  the 
variables,  and  the  number  of  correctly  and  wrongly  predicted  preferences 
are  almost  the  same  in  the  corresponding  pairs  of  tables. 
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Table  XIII 

Comparison  of  actual  preferences  for  the 
entire  layout  versus  predicted  preferences 
using  confidence  coefficients  of  1  and  1 


Block  C 


Attribute 

Layout 
Pair  (1) 

Confidence  Layout 
Level  (2)  preferred 
by  DM 

Prediction  Prediction  of 
of  model  (3)  model  consid¬ 
ering  "certain" 
subscores  (3) 

Noise 

AB 

1 

B 

-1 

-1 

BC 

1 

C 

1 

1 

CD 

1 

C 

1 

1 

AC 

1 

C 

1 

1 

AD 

1 

D 

-1 

-1 

BD 

2 

D 

1 

1 

Temperature 

CD 

1 

D 

1 

1 

AD 

1 

D 

-1 

-1 

BD 

1 

D 

1 

1 

Humi di ty 

CD 

1 

D 

-1 

-1 

AD 

1 

D 

-1 

-1 

BD 


1 


D 


-1 


-1 
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Functional 

AB 

2 

B 

-1 

-1 

BC 

2 

C 

1 

1 

AC 

1 

C 

1 

1 

AD 

1 

D 

1 

1 

BD 

2 

D 

1 

1 

Block  D 

(equation 

(5.9)) 

Attributes 

considered 

BC 

1 

C 

1 

1 

simul- 

AC 

1 

C 

-1 

-1 

taneously 

AD 

1 

D 

-1 

-1 

BD 

1 

D 

1 

1 

Number  of  correct  and  wrong 
predictions  considering  attributes 

simultaneously  (Block  0)  +2,-2  +2,-2 

Number  of  correct  and  wrong 
predictions  considering  attributes 

separately  (Block  C)  +10,-7  +10,  -  7 


(1)  A,  B,  C,  D  were  four  layouts  used  for  pairwise  comparisons 

(2)  1:  certain  2:  reasonably  sure 

(3)  1:  correct  prediction  -1:  wrong  prediction 
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6.0  CONCLUSIONS 

Detailed  conclusions  have  been  stated  in  various  sections  of 
this  report  and  various  problem  areas  have  been  highlighted.  The 
summary  conclusion  that  can  be  drawn  is  that  the  model  does  seem  to 
give  encouraging  results  although  it  has  not  yet  undergone  sufficient 
validation  testing. 

One  key  step  toward  further  validation  is  that  of  improving  the 
questionnaire  interface  with  the  DM.  The  "hard  copy"  questionnaire 
[1]  is  simply  too  cumbersome  with  volunteer  surrogate  decision-makers 
Toward  such  an  improvement,  the  consideration  of  an  interactive 
computer-based  data  development  system  seems  especially  promising, 
particularly  when  coupled  with  graphics  displays  at  the  user  terminal 
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